The present invention generally relates to the storage and retrieval of electronic data in a computer managed data archive. More specifically, the present invention provides for the efficient storage and retrieval of audio and video data suitable for distribution over a broadcasting network in a media lifecycle management system.
Broadcast facilities employ a wide variety of electronic equipment to receive, process and transmit audio-visual content to audiences. A key component in a broadcast content delivery system is a data storage archive and media lifecycle management system where audio-visual content is stored both before and after it is transmitted for broadcast. Data storage archives can include a wide variety of storage media and associated hardware such as magnetic tape, optical disk and hard disk drive arrays. The various data storage devices used within a broadcasting facility are interconnected, via a computer network, to supply devices which receive data from external sources. Examples of these supply devices includes production equipment which prepare and edit broadcast material and video servers which are used to play-out material to broadcast transmitters. A distinguishing characteristic of a data archive and media lifecycle management system, compared with a typical computer network, is the tremendous amount of data that constitutes broadcast quality video when it is converted to a digital form and the ongoing need for transporting such large volumes of data among various devices within the system in a timely manner.
The various storage hardware and media have different performance and cost characteristics which often determines how they are used within a broadcast content system. Video servers, which feature high data transfers speeds and are relatively expensive, are used as play-out devices to transfer electronic video data to broadcast transmitters. On the other end of the performance/cost spectrum are analog magnetic tapes which are inexpensive but limited in data access speed. There is a similar trade off in performance versus cost for the data channels and associated transmission protocols which are used to connect the various storage devices with each other. Also, trade-offs exist for the types of connections used to interconnect to the computer(s) which control the overall broadcast content management system. At the low end of the spectrum of performance is ethernet cable used in standard computer networks to connect network servers to client workstations which is readily available at low cost. A typical high performance data channel technology is a fiber channel, which is faster but commensurately more expensive. Based on these options/trade-offs, it is desirable to construct a data archive and media lifecycle management system with an optimal blend of performance and cost for both data storage and data transmission hardware.
In addition to employing a cost effective architecture of storage and transmission hardware, the media lifecycle management systems must also be reliable and flexible. Broadcast content typically includes both program content for audiences as well as commercial advertising which is a major source of revenue for broadcasters. Any system down time will negatively impact a broadcaster's immediate revenue as well as their reputation with audience members. It is important therefore that the media lifecycle management system be reliable and robust on both a component level as well as on an overall system level. This includes a consideration of the future growth and maintenance of the system. The ability to make changes to the system, without interruption in operation, is especially important at a time like today when rapid advances are being made in computer and related digital storage technologies.
In a Referring now to FIG. 1, a prior art data archive and media lifecycle management system 1 is shown schematically. This system illustrates the traditional approach to media data management that has been employed to date. The system is comprised of various components which cooperate to perform the tasks of storing, indexing, searching, and retrieving data as needed in day to day operations. A centralized system core 2 contains software which communicates with and controls external data storage devices such as a digital video and/or media servers (DVS) 6, a system memory cache 7, and a media library 8. The media library is shown having a number of separate memory modules which represent an organized plurality of devices such as magnetic tape drives, hard disk drives, optical drives, holographic drives or any other suitable data storage devices. System core 2 is also connected to an external database 3, which contains a searchable index of the storage locations of all data stored by the system. Typically the index records in the database will contain descriptive metadata related to the data stored on the various external storage components 6-8. In addition, database 3 contains an active database of the configuration of the data archive and media lifecycle management system 1 as well as operation and status logs. The system core 2 is connected to the various external components through two separate data channels. The dashed lines 9 represent internet protocol (IP) infrastructure and could be either Ethernet or GigEthernet which are capable of data transfer rates of 10-100 Mbits/sec for the former and 1 Gbit/sec for the latter. The solid lines 10 represent Fiber Channel (FC) which have a significantly higher data transfer capacity in the range of 1-4 Gbits/sec. Expected advances in FC technology will lead to yet higher data transfer capacities of 8 Gbits/sec within several years and a further increase to 16 Gbits/sec several years after that.
Several approaches to data storage and media lifecycle issues have been developed in the past. Several of these include:
U.S. Pat. No. 4,296,465 discloses a method of moving substantial blocks of data from an I/O buffer or disk cache buffer disk (external) memory to working processor memory for use by a program running in the computer system. The method involves the use of an autonomous hardware device, a so-called data mover, to affect the transfer of data. The data mover can receive and store instruction words and data words and is provided with the necessary registers, counters, switches and digital logic circuits so that it can issue appropriate memory retrieval and storage (read and write) instructions. The data mover is provided with a throttle to minimize potential conflicts with other components of the data processing system. It is designed so that its read and write instructions overlap each other, significantly increasing the rate of data transfer. The data mover relieves the control processors of the data processing system of the responsibility of moving blocks of data from the I/O buffer to the working store thereby significantly increasing the rate of data transfer.
U.S. Pat. No. 5,091,849 describes a computer image production system that features computer workstations connected to data image storage devices which contain image data that is retrieved and processed by the workstations. The workstations are connected to the storage devices by two separate communication networks; a first network for sending and receiving system logistics and control information to the data storage devices and a second network for transferring data files from storage to the workstations and vice versa.
U.S. Pat. No. 5,860,026 involves a method of transferring data between clusters in a system comprising a plurality of clusters by issuing data transfer instructions from a first cluster which are carried out by a data transfer processing unit on a second cluster without having to use a main data processor in the second cluster. This allows the system to transfer data between clusters without compromising the general data processing ability of clusters affected by the data transfer.
U.S. Pat. No. 5,917,723 discloses a method of mirroring write request data from a first device to a second device where the second device has a primary and a secondary processor. Variable length control data is included in the data stream to the second device and is received by the secondary processor which reads and executes the control data without requiring a hardware interrupt of the primary processor of the second device.
U.S. Pat. No. 5,944,789 involves a network file server consisting of a cached disk array and a plurality of data mover computers for providing access to data files stored in the cached disk array. The data movers perform file system tasks such as mapping file names to logical data blocks, and the locking and unlocking of data files to minimize the loading on the cached disk array. Each data mover maintains a local cache of the file directory software including locking information of locked files that are accessible through the data mover. Data transfers may use more than one data mover simultaneously to expedite the transfer of data from the cached disk array to a client computer. A cache consistency scheme ensures that file locking information is consistent among the local caches of the plurality of data mover computers.
U.S. Pat. No. 6,324,581 describes a file server system in which a plurality of data mover computers control access to data stored on respective data storage devices. A network client can access data stored in the system through any of the data movers. If a data mover receives a request for data whose access is controlled by another data mover, the second data mover performs a lock on the requested data and sends metadata for the requested data to the first data mover computer which then uses the metadata to formulate a data access command to retrieve the requested data over a data path which bypasses the data mover computer which controls access to the requested data.
U.S. Pat. No. 6,651,130 outlines a computer system interface between a host computer/server and a storage device comprising an array of disk drives. The interface includes a plurality of front end directors coupled to the host computer server and a plurality of back end directors coupled to the array of disk drives. The interface also comprises a data transfer section containing a cache memory and a messaging network that is used to send control signals between the front and back end directors. Each director includes a data pipe for transmitting data to and from the cache memory in the data transfer section, a microprocessor, a controller and a common bus for connecting the aforementioned components. The controller manages the exchange of messages between the front and back end directors used to control the transfer of data through the data pipes to the cache memory in the data transfer section.
U.S. Pat. No. 6,735,717 describes a distributed computing clustering model which features at least two file servers each of which uses a virtual shared memory containing data objects or tuples which can be accessed independently by worker processes running on client computer devices. A state table is used to synchronize the active virtual shared memory of a first active file server with the stand-by virtual memory of the stand-by file server. In the event of a failure, the stand-by file server will become active and the worker processes will be able to access data objects or tuples in its version of the shared virtual memory.
U.S. Pat. No. 6,779,071 discloses a computer interface between a host computer/server and a storage device comprising an array of disk drives. The interface includes a plurality of front end directors coupled to the host computer server and a plurality of back end directors coupled to the array of disk drives. The interface also comprises two independent communication networks, one for transmitting control messages between the front and back end directors and a second for transmitting data through a common cache memory between the front and back end directors.
U.S. Pat. No. 6,990,606 involves a method for effecting a failover change of control from one node in a loosely connected cluster of nodes in a distributed computer environment to another candidate failover node. Configuration data for a plurality of candidate failover nodes is maintained in a data store and status messages are sent from each of the candidate failover nodes, analyzed and compared with the configuration information to determine if a failure in the current controlling node has occurred and if a failover change of control should be initiated. The method provides for a cascading failover which provides for a first failover candidate taking over only a portion of the processes of a failed control node with a second failover candidate taking over the remaining functions.
U.S. Pat. No. 7,010,575 describes a method of transferring data between a host computer and a bank of disk drives over an interface which comprises two separate communication networks connecting front end data transfer directors connected with the host computer and back end data transfer directors connected with the bank of disk drives. The interface mediates and controls the transfer of data from the host computer to the disk drives and vice versa. One of the communication networks in the interface is used to pass control messages from the front end directors to the back end directors and vice versa while the second communication network is reserved for the transfer of data only from the host computer to the disk drives or vice versa through an intermediary cache memory. The use of a separate network for messaging relieves the cache memory from the burden of storing and transmitting the control messages and improves the operation of the system. The method includes the provision of checking where messages received by a second director are from a valid first director, and if not, terminating the data transfer transaction.
U.S. Pat. No. 7,028,218 illustrates a redundant file server system which uses a plurality of processor boards each having at least 2 physical processors each of which share a cache memory, multiple functional units and a common register unit allowing each of the physical processors to be programmed to act as separate logical processors. The separate logical processors on each processor board are programmed to function as network control processors, data storage control processors and data mover processors in either an active functioning mode or in a stand-by mode ready to take over from the active processor in the event of a failure.